Config hygiene: validate parameter, flag conflicting flags, dedup source list#101
Merged
Conversation
…rce list Three low-risk improvements to make config mistakes surface early instead of failing silently or far downstream: - validate() now rejects an unknown `parameter` (guarded so empty-parameter sites-only flows still pass), and emits an advisory warning when more than one spatial filter (bbox/county/wkt) or more than one output mode is set — states the code currently accepts but resolves inconsistently. - Orchestration's DIEConfigResource no longer hardcodes the source list for include-list products; it derives from backend.config.SOURCE_KEYS, so a new source can't be silently dropped (removes the "must stay in sync" note). Adds tests/test_config_validation.py. Full suite (297) + dg check defs clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Your pull request is automatically being deployed to Dagster Cloud.
|
This was referenced Jun 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First of two config-system improvements (the quick, low-risk wins). Makes config mistakes surface early instead of failing silently or far downstream.
Changes
Config.validate()now:parameter(SystemExit, consistent with the existing bbox/county/date checks) — guarded so empty-parameter sites-only flows still pass.bbox/county/wkt) is set — these resolve with different precedence inbbox_bounding_points(bbox first) vsbounding_wkt(wkt first), so >1 silently does different things.DIEConfigResourcederives the include-list source set frombackend.config.SOURCE_KEYSinstead of a hardcoded list — kills the "must stay in sync with SOURCE_KEYS" duplication so a new source can't be silently dropped from an include-list product.Verification
tests/test_config_validation.py(new, 9 tests).dg check defsclean.Advisory checks are warn-only — no behavior change for valid configs. Follow-up PR introduces a single source registry to collapse the
SOURCE_DICT+ two*_SOURCE_PAIRStables.🤖 Generated with Claude Code